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TELEMAMMOGRAPHY  SYSTEM:  A  MODEL  FOR  A 
REGIONAL  SYSTEM 

ELLEN  SHAW  DE  PAREDES,  M.D. 

SAMUEL  J.  DWYER,  Ph.D. 

MEDICAL  COLLEGE  OF  VIRGINIA,  RICHMOND,  VA 
UNIVERSITY  OF  VIRGINIA,  CHARLOTTESVILLE,  VA 


PURPOSE: 

The  research  hypothesis  being  tested  is  that  a  telemammography  system  provides  a 
mechanism  for  digitizing,  transmitting,  archiving,  and  displaying  mammograms  so  that  a  trained 
radiologist  utilizing  grayscale  monitors  (2k  x  2k  x  8/12  bits)  can  interpret  the  images  with  an 
accuracy  level  sufficient  for  primary  diagnosis.  A  new  metric  for  measuring  the  performance  of 
telemammography  systems  has  been  developed,  throughput/cost  ratio.  This  metric  provides  a 
measure  for  comparing  analog  to  digital  mammography  systems.  This  metric  has  been  employed 
in  comparing  manufacturing  processes,  i.e.  jobs/sec/cost  (1).  A  ROC  curve  analysis  (2)  is 
utilized  to  compare  the  accuracy  of  interpretation  of  analog  images  versus  digitized  images. 

The  hypothesis  is  being  tested  by  utilizing  a  laser  film  digitizer  with  a  50-micron  pixel  size 
and  by  performing  ROC  studies  to  compare  conventional  analog  screen-film  mammography  with 
digitized  screen-film  mammograms  displayed  on  grayscale  workstations  (two  monitors,  each  2k 
X  2k  X  8/12  bits).  The  wide  area  network  (WAN)  being  utilized  are  terrestrial  and  satellites  links. 
The  goal  of  this  study  is  to  determine  the  requirements  to  deliver  high  quality,  high  resolution 
mammography  images  from  remote  locations  that  may  not  have  a  terrestrial  data 
communications  infrastructure  available. 

DESCRIPTION  OF  EXPERIMENTAL  PROCEDURES: 

A  series  of  retrospectively  collected  200  normal  mammograms  and  200  abnormal 
mammograms  containing  a  single  lesion  including  masses,  calcifications,  dilated  lactiferous 
ducts,  and  asymmetric  densities  were  collected  for  conducting  a  ROC  analysis.  The 
mammographic  and  pathologic  findings  were  classified  using  the  terminology  of  the  American 
College  of  Radiology  Lexicon.  The  ROC  study,  which  used  12  readers  compares  analog  screen- 
film  mammograms  and  the  mammograms  which  were  digitized  at  50  microns  spot  size.  The 
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digitized  analog  screen-film  mammograms  were  archived  into  a  database.  A  comparison  ROC 
curve  analysis  of  the  digitized  mammograms  is  being  conducted  by  displaying  them  on  a  2k  x  2k 
X  8/12  bit  grayscale  workstation  using  two  monitors.  An  evaluation  performance  is  being 
conducted  using  the  metrics  of  throughput  and  throughput-to-cost-ratio.  The  throughput  and 
throughput/cost  ratio  is  based  on  the  method  of  Jackson  Network  Analysis  (3)  and  has  been 
completed  on  the  analog  system.  The  calculations  are  based  on  a  resource  allocation  table  in 
which  the  average  time  required  for  each  step  is  measured  as  applicable  for  each  resource.  A 
satellite  transmission  of  mammograms  is  underway  between  the  University  of  Virginia,  the 
Cleveland  Clinic  Foundation,  and  the  NASA  Lewis  Research  Center  (ACTS  Project). 

A  soflcopy  display  protocol  has  been  developed  and  is  being  implemented  to  review  the 
images  in  a  similar  format  to  the  manner  in  which  they  are  interpreted  on  a  viewbox 

SUMMARY  OF  RESULTS  TO  DATE: 

An  ROC  analysis  has  been  conducted  on  the  400  analog  screen  film  mammograms.  For  all 
readers  the  results  of  the  ROC  analog  images  varied  10%  from  the  0.83  area  under  the  ROC 
curve,  which  is  a  desirable  result,  reflecting  proper  case  selection.  A  ROC  analysis  is  underway 
for  the  two  monitor,  grayscale  2k  x  2k  workstation  display  of  the  digitized  mammograms.  The 
results  of  the  throughput  analysis  of  the  analog  images  showed  that  the  technologist  was  the 
bottleneck  (0.041  jobs/minute)  and  the  throughput  to  cost  ratio  was  0.001203.  Our  measure  of 
luminance  of  the  workstation  shows  the  average  luminance  level  to  be  1/1  0  that  of  the 
mammography  viewbox.  The  contrast  range  for  the  grayscale  display  is  20%  lower  than  that  of 
analog  images. 

CONCLUSIONS: 

The  resulting  soflcopy  display  protocol  for  the  digitized  screen  film  mammograms  reduces 
the  throughput  by  a  factor  of  one-half.  The  digitized  screen  film  images  (4k  x  4k  x  12  bits) 
presents  a  difficult  display  protocol  for  a  2k  x  2k  x  8/12  bits  monitor.  Additional  challenges  to 
display  are  the  presentation  of  4  images  on  2  monitors  as  well  as  the  review  of  prior  images  for 
comparison.  Because  of  the  lower  contrast  range  of  the  grayscale  displays,  image  processing 
(window  and  level  functions)  is  needed  on  the  soflcopy. 

METHODS: 

Image  Selection. 

A  database  of  200  normal  screen-film  mammography  examinations 
and  200  abnormal  screen-film  mammography  examinations  have  been  collected. 

Each  examination  contains  the  following:  image  study  performed;  results;  description  of 
mammographic  findings  if  abnormal  (the  same  as  the  BIRADS);  type  and  report  of  biopsy 
performed;  demographics;  procedural  and  diagnostic  radiation  exposure;  and  clinical  follow-up. 

The  abnormal  films  comprise  a  representative  sample  of  pathological  diagnosed  breast  lesions  in 
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the  population  served  by  the  University  of  Virginia  and  MCV.  The  normal  films  have  been 
substantiated  by  clinieal  follow-up  and  are  age  matched  to  the  overall  parenchymal  density  of  the 
abnormal  films.  All  patient  data  and  names  were  blocked  and  a  case  number  assigned. 


DIGITIZED  IMAGE  DATABASE 

The  collected  database  of  analog  mammographic  films  was  converted  into  digital  arrays 
using  a  laser  film  digitizer  with  a  50  micron  pixel  spot  size  and  12  bits  per  pixel  (Lumisys, 
Sunnyvale,  CA,  Model  150  with  a  50-micron  spot  size).  The  laser  film  digitizer  was  connected 
to  a  sun  workstation  9SPARC  Model  40)  and  connected  to  an  internet  connectivity.  All  patient 
data  were  removed  and  a  case  number  assigned  to  each  digitized  case.  This  image  database  is 
being  recorded  on  a  CD-ROM  for  use  by  other  researchers.  The  digitized  images  are  archived 
into  a  database  and  archived  on  both  a  hard  drive  disk  and  a  Tape  drive. 

The  laser  film  digitizer  uses  an  analog  film  SMPTE  pattern  once  a  week  to  provide  an 
adequate  quality  control.  Additionally,  the  laser  film  digitizer  is  calibrated  once  a  week  using  the 
protocol  provided  by  the  manufacturer.  Each  digitized  image  was  displayed  on  a  grayscale 
display  workstation  to  assure  proper  digitization  of  the  screen-film  images. 

IMAGE  GRADING  AND  READING. 

The  readers  use  a  5-point  graded  response.  Each  reader  completed  a  worksheet  form  for 
each  image,  requiring  a  response  to  the  following  six  questions  by  circling  the  selected  graded 
response.  The  pathology  reports  of  the  biopsies  served  as  ground  truth. 

Masses: 

1 .  (definitely  not  present):  2.  (probably  not  present); 

3.  (equivocal;  4.  (probably  present);  5.  (definitely  present) 

Microcalcifications: 

1 .  (definitely  not  present);  2.  (probably  not  present); 

3.  (equivocal);  4.  (probably  present);  5.  (definitely  present). 

Dilated  lactiferous  ducts: 

1 .  (definitely  not  present);  2.  (probably  not  present); 

3.  (equivocal);  4.  (probably  present);  5.  (definitely  present). 

Focal  areas  of  asymmetry  or  architectural  distortion: 

1 .  (definitely  not  present);  2.  (probably  not  present); 

3.  (equivocal);  4.  (probably  present);  5.  (definitely  present). 

Thickening  or  retraction  of  the  skin; 
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1 .  (definitely  not  present);  2.  (probably  not  present): 

3.  (equivocal);  4.  (probably  present);  5.  (definitely  present). 

Diagnosis  of  image: 

1 .  (definitely  benign);  2.  (probably  benign); 

3.  (equivocal);  4.  (probably  malignant);  5.  (definitely  malignant) 

Images  were  presented  to  the  set  of  readers  first  as  analog  mammographic  films.  The 
films  were  hung  in  sets  on  several  rollerscopes  in  the  department.  The  readers  read  each  image 
and  responded  to  the  above  six  questions.  The  readers  are  just  now  starting  to  read  the  digitized 
film  images  on  a  grayscale  workstation.  It  is  believed  that  enough  time  has  passed  from  the 
analog  screen  film  readings  that  the  readers  will  not  remember  the  cases.  The  readers  may  utilize 
a  magnifying  glass  as  an  option  if  they  wish.  A  random  number  generator  is  being  used  to 
modify  the  order  of  presentation  on  the  grayscale  display  as  compared  to  the  presentation  of  the 
analog  mammography  images.  Reader  data  is  recorded  in  a  notebook,  one  page  per 
mammographic  exam.  The  reader  data  is  then  entered  into  a  database  and  a  ROC  analysis 
performed  for  each  of  the  six  questions  responded  to  by  the  readers. 

STATISTICAL  CONSIDERATIONS 

In  selecting  the  sample  size,  we  have  used  two  methods  to  increase  the  reliability  of  the 
calculation.  In  the  first  method,  we  have  chosen  type  I  error  to  be  equal  to  0.05  (1 .96)  for  a  two 
tailed  test  and  type  1 1  error  to  be  equal  to  0.20  (0.84).  Hence  the  power  index  (PI)  is  2.8.  The 
sample  size  is  given  by  n  equal  to  ((PI  x  SE/  (A1  -  A2))  squared  where  A1  is  the  area  under  the 
ROC  curves  for  analog  films  and  A2  is  the  area  under  the  ROC  curves  for  digitized  and  grayscale 
displayed  images.  We  estimate  the  standard  error  to  be  0.5  for  12  readers,  all  providing  primary 
coverage  in  mammography.  We  assume  the  differences  in  the  means  of  the  areas  under  the  two 
ROC  curves  to  be  10%.  Then  the  number  of  samples  per  population  is  196  or  approximately  200 
pairs  of  images.  Hence  we  will  require  200  normal  and  200  abnormal  mammograms. 

In  a  second  method  that  confirms  the  findings  of  the  first  approach,  we  assume  the 
following;  alpha  error  0.05,  beta  error  0.2,  power  80%,  and  the  expected  difference  between  the 
areas  underneath  the  ROC  curves  for  the  digital  (A2),  and  analog  (Al)  mammographic  films  as 
0.1.  This  expected  difference  is  the  minimum  difference  that  we  consider  clinically  important. 
Specifically,  we  are  estimating  Al  to  be  0.85  and  A2  to  be  0.75.  Since  each  reader  will  review  the 
same  sample  of  cases,  we  will  use  a  correlation  coefficient  conservatively  estimated  at  0.3.  Using 
the  methods  of  Hanley  and  McNeil  and  a  two-sided  test  of  significance,  we  calculate  that  with  a 
single  reader  we  will  require  120  cases  each  of  abnormal  and  normal  films.  Thus,  if  the 
difference  between  Al  and  A2  is  as  estimated  above,  we  will  have  a  greater  than  80%  chance 
with  a  single  reader  of  detecting  a  statistically  significant  difference  in  diagnostic  performance 
with  120  pairs  of  normal  and  abnormal  cases.  Using  additional  readers  (12  total)  and  cases  (200 
each  of  normal  and  abnormal)  will  give  us  a  greater  likelihood  of  detecting  a  significant 
difference. 
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A  ROC  curve  analysis  of  the  reader  data  is  accomplished  by  using  applications  software 
provided  by  Charles  Metz  of  the  University  of  Chicago/University  of  Piftsburgh.  The  software  is 
operational  on  a  SUN  workstation  at  UVA.  Two  ROC  curves  for  each  reader  for  each  of  the  six 
reader  questions  will  be  generated,  one  for  the  analog  mammographic  films  and  one  for  the 
digitized  mammograms  displayed  on  the  grayscale  display  workstations  on  our  AGFA  image 
display  workstation.  The  index  of  performance  will  be  the  area  under  the  ROC  curve  for  each 
reader.  In  the  event  of  a  degenerate  ROC  curve,  we  analyzed  the  data  using  two  approaches. 

One  approach  utilized  the  Wilcoxon  statistic  to  estimate  the  area  underneath  the  ROC  curve  and 
the  standard  error.  The  second  approach  involved  a  third-order  spline  curve-fitting  software 
program  implemented  at  the  University  of  Virginia. 

MAMMOGRAPHY  GRAYSCALE  READING  WORKSTATION 

The  grayscale  workstation  for  the  project  is  a  AGFA  review  station  (IMP AX,  RS3000, 
Ridgefield  NJ)  which  is  comprised  of  a  SUN  Ultra  2  with  192  Mbytes  of  RAM,  4  Gbytes  of 
internal  HD  storage,  two  DOME  high-resolution  video  cards,  and  two  high-resolution  (2k  x  2.5k) 
Orwin  grayscale  monitors.  The  system  is  running  Solaris  2.5,  with  Common  Desktop 
Environment  installed  and  uses  the  Motif  Graphical  User  Interface.  The  Review  station  software 
is  DICOM  3.0  compliant. 

We  added  a  Radion  4Gbyte  external  hard  disk  drive  for  storing  images  from  the  current 
reading  set,  and  a  Pinnacle  Micro  Vertex  2.3  Gbyte  Magneto-Optical  drive  as  a  transfer/backup 
device. 

A  separate  instance  of  the  SMV  software  was  started  on  the  desktop  of  each  monitor,  and 
expanded  to  full  screen.  This  allowed  two  images  to  be  displayed  at  once.  Initially,  the  LMLO 
image  was  displayed  on  the  left  monitor,  and  the  RMLO  was  displayed  on  the  right  monitor. 

This  was  followed  by  the  LCC  and  the  RCC,  respectively,  and  then  images  were  displayed  as 
requested  by  the  radiologist  reader. 

DISPLAY  WORKSTATION  SOFTWARE 

The  software  being  used  is  SMV,  a  program  wriften  by  Prof  Marty  Stanton  of  Brandeis 
University.  Originally  designed  for  crystallography  work.  Prof  Stanton  adapted  it  to  the  needs  of 
the  Telemammography  project  by  giving  it  the  ability  to  directly  read  the  images  created  by  the 
Lumisys  150  laser  film  digitizer.  This  made  it  possible  to  display  the  images  without  going 
through  the  intermediate  step  of  converting  our  data  to  the  DICOM  format  needed  by  the 
IMP  AX  database  of  the  AGFA  review  station. 

SMV  supports  all  of  the  image  processing  techniques  needed  for  the  display  of  the 
mammography  images,  including  panning,  zooming,  window-leveling,  and  grayscale  inverting. 

It  also  provides  features  that  were  used  extensively  to  prepare  the  images  for  display,  particularly 


5 


« 


rotation  and  cropping. 

Two  factors  had  to  be  considered  when  preparing  the  images  for  display.  One  was  that 
patient  information  recorded  on  the  mammography  film,  as  well  as  indication  of  the  source 
hospital  for  the  films,  had  to  be  masked.  The  other  was  that  a  reduction  in  the  overall  size  of  the 
images  was  desirable,  to  decrease  storage  requirements  and  reduce  the  amount  of  time  required 
to  load  and  display  the  image.  To  address  both  of  these  concerns,  SMV's  cropping  feature  was 
used  to  reduce  the  image  to  just  the  part  of  the  film  that  actually  held  the  image  of  the  breast. 

This  greatly  improved  performance  of  the  system,  as  well  as  satisfying  the  protocol  requirement 
that  no  patient  or  hospital  information  be  visible  to  the  reader. 

DISPLAY  PROTOCOLS  FOR  WORKSTATION 

The  Display  Protocol  for  the  AGFA  workstation  consists  of  four  displays;  Display  A  (LMLO 
on  left  screen  and  RMLO  on  right  screen);  Display  B  (LCC  on  left  screen  and  RCC  on  right  screen); 
Display  C  (LMLO  on  left  screen  and  LCC  on  right  screen);  and  Display  D  (RMLO  on  left  screen 
and  RCC  on  right  screen).  Display  A  and  B  were  presented  to  the  reader  and  if  requested  then 
Display  C  and  Display  D  were  also  presented. 

FUTURE  EFFORTS 

During  the  Fourth  Year  of  the  Contract,  we  will  accomplish  the  following: 

1 .  Complete  reading  the  400  cases  on  the  AGFA  grayscale  workstations 

2.  Complete  the  ROC  analysis,  comparing  screen-film  readings  with  those  of  the 
grayscale  (softcopy)  for  all  12  readers 

3.  Complete  the  NASA  ACTS  Telemammography  readings 

4.  Complete  and  evaluate  the  Wide  Area  Network  connection  between  Stony  Point  and 
MCV  and  testing  the  link  with  the  400  digitized  film  images.  We  plan  to  use  a  T1  and 
a  FRAME  RELAY  connection. 

5.  Complete  the  Throughput  Workflow  Analysis 

NASA-LERC  ACTS  TELEMAMMOGRAPHY  PROJECT 

The  ACTS  telemammography  project  a  joint  effort  between  NASA's  Lewis  Research 
Center,  the  Cleveland  Clinic,  and  the  University  of  Virginia  Department  of  Radiology.  ACTS  is 
NASA's  Advanced  Communications  Technology  Satellite.  Earth  stations  have  been  installed  at 
the  three  locations.  Unix  workstations  are  connected  to  the  earth  stations  at  each  location.  The 
purpose  of  the  project  is  to  test  the  efficacy  of  using  a  satellite  link  as  a  means  of  transmitting 


6 


*  • 


mammography  images  to  distant  locations.  The  experimenters  are  transmitting  digitized 
mammography  images  via  ACTS  satellite  between  UVa,  NASA-LERC  and  the  Cleveland  Clinic 
with  various  techniques  and  levels  of  compression.  Radiologists  at  Cleveland  Clinic  are 
examining  the  images  to  determine  their  quality  and  diagnostic  effectiveness.  The  current  phase 
of  this  project  is  expected  to  run  through  the  spring  of  1998. 

THROUGHPUT  ANALYSIS 

The  performance  metrics  being  used  in  our  Telemammography  modeling  are  diagnostic 
accuracy  and  throughput.  Diagnostic  accuracy  is  obtained  by  a  ROC  analysis.  The  area  under 
the  ROC  curve  is  the  probability  of  correctlly  distinguishing  -  on  the  basis  of  the  diagnostic  test 
results  alone  -  a  randomly  selected  disease  case  from  a  randomly  selected  non-diseased  case. 
Throughput  is  defined  as  the  rate  jobs-per-minute)  at  which  requests  can  be  serviced  by  the 
system.  The  throughput  of  a  system  generally  increases  as  the  load  on  the  system  initially 
increases,  without  the  addition  of  more  resources;  after  a  certain  load,  the  throughput  stops 
increasing  due  to  one  or  more  bottlenecks.  Throughput  analysis  is  accomplished  by  listing  the 
resources  used  and  the  steps  required  to  complete  one  job  together  with  the  average  time  per  step. 
The  resource  with  the  smallest  jobs-per-minute  over  the  complete  job,  is  termed  the  bottleneck 
resource. 

Table  1  and  Table  2  are  the  resource  utilization  tables  used  for  conducting  throughput 
analysis,  generated  for  MCV  (Medical  College  of  Virginia)  and  Stony  Point  (an  MCV  outpatient 
clinic),  both  in  the  Breast  Imaging  section  of  the  Department.  The  resources  being  modeled  in 
Table  1  and  Table  2  are:  The  Clerk,  the  Technologist,  the  Imaging  modality,  the  Film  Processor, 
the  File  Room,  the  Resident,  and  the  Radiologist.  At  each  step,  the  average  time  to  complete  the 
step  is  measured  (minutes).  The  Throughput  for  each  resource  is  calculated  and  recorded  at  the 
bottom  of  each  table.  For  example,  in  Table  1,  the  throughput  of  the  Tech  is  calculated  by  1/(13.2 
+  4.7  +  1.6  +  5.13)  =  0.041  jobs-per-minute.  This  calculation  assumes  that  the  Tech  is  working 
at  100%  capacity  and  can  work  no  faster.  In  Table  1,  the  system  bottleneck  is  the  Tech  since 
0.041  jobs-per-minute  is  the  smallest  throughput.  Likewise,  in  Table  2,  the  Tech  is  again  the 
bottleneck  at  0.071  jobs-per-minute. 

One  of  the  tasks  to  be  accomplished  is  the  use  of  modeling  techniques  for  a  regional 
telemammography  system.  This  section  of  our  report  describes  the  modeling  schemes  we  have 
developed. 

INTRODUCTION 

Mammography  is  used  routinely  to  screen  for  occult  breast  cancer  and  is  an  essential 
element  in  the  health  care  service  for  adult  women.  Screening  mammography  is  a  high  volume 
radiographic  procedure  with  less  than  one-percent  of  the  cases  demonstrating  breast  cancer.  Yet, 
in  retrospect,  radiologists  do  not  detect  all  cancers  visible  on  the  radiographic  images.  Missed 
detections  are  due  to  such  factors  as  low  conspicuity  of  the  lesion,  poor  image  quality. 
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radiologists’  fatigue,  and  oversights  by  the  radiologists  (1).  Screen-film  mammography  is  the 
imaging  modality  most  often  used  (2-5),  but  Full  Breast  Digital  Mammography  (FBDM)  is 
current  printers  are  limited  to  printing  arrays  of  only  4k  X  5K  x  12  bit  on  8  X  10  or  10  X  12  inch 
detection,  high  contrast  resolution  over  a  large  dynamic  range,  and  high  detector  quantum 
efficiencies  for  x-rays.  The  digital  technology  of  FBDMs  also  provides  the  capability  to  separate 
and  optimize  the  functions  of  image  generation  and  image  display.  In  addition,  FBDM  systems 
offer  improved  access  to  digital  technology,  with  its  promise  of  inexpensive  telemammography 
and  of  using  newly  developing  digital  data  management  systems.  The  challenge  for  FBDM 
systems  is  managing  and  displaying  the  extremely  large  amount  of  digital  image  data  generated. 
These  systems  generate  a  4800  x  6400  x  14-bit  digital  array  per  image.  The  data  per  screening 
examination  is  215  Mbytes  for  four  standard  screening  views  (craniocaudal  (CC)  view,  left  and 
right  breast;  mediolateral  oblique  view  (MLO),  left  and  right  breast).  In  comparing  the  new 
examination  to  the  previous  examinations,  a  total  of  over  430  Mbytes  of  image  data  needs  to  be 
displayed  efficiently.  Two  choices  for  display  technologies  are:  (a)  high  resolution  laser  film 
printers;  and  (b)  interactive  grayscale  workstations. 

The  limitations  regarding  the  choice  for  FBDM  image  display  are  significant.  Laser  film 
printers  are  difficult  to  utilize  because  optimized  look-up  tables  for  each  image  are  needed  and 
current  printers  are  limitd  to  printing  arrays  of  only  4k  x  5K  x  12  bit  on  8  x  10  or  10  x  12  inch 
film.  On  the  other  hand,  interactive  grayscale  workstations  display  only  a  2k  x  2.5k  at  8/12  bits 
(12  bits  from  a  frame  buffer)  per  monitor,  require  complex  display  protocols  (display  and 
peregrinating  through  8  images,  each  6.4k  x  4.8k  x  12  bits,  viewed  by  a  2k  x  2.5k  display  screen 
per  monitor).  The  usual  choice,  due  to  cost,  is  two  monitors  per  workstation.  Thus  there  are 
tradeoffs  between  these  two  display  technologies. 

The  advantages  of  the  high  resolution  laser  film  printer  are  the  following:  (a)  the  matrix 
size  of  the  laser  printed  digital  mammography  images  (5k  x  4k)  comes  close  to  matching  that  of 
the  FBDM  image  (6.4k  x  4.8k);  (b)  the  size  of  the  laser  film  printed  image  matches  that  of  the 
screen-film  size  of  8x10  or  10x12  inches  and  (c)  once  printed  and  processed,  the  laser  film 
printed  image  can  be  displayed  on  mammography  view  boxes  and  managed  in  the  same  manner 
as  are  standard  screen-film  images.  The  disadvantages  of  the  laser  film  printer  are  the  following: 
(a)  requires  20  seconds  per  image  for  exposing  the  latent  image  (prints  1  line  per  2.2  msec)  and 
then  the  standard  90  seconds  to  develop  the  film  before  clinical  interpretation  of  each  image  is 
possible;  (b)  look-up  tables  must  be  developed  to  compensate  for  nonlinear  optical  characteristics 
of  film;  in  addition,  because  the  dynamic  range  of  film  (optical  density  form  0.2  up  to  3.0  or  less) 
is  much  less  than  that  of  digital  mammographic  detectors,  dynamic  range  compression 
algorithms  and/or  multiple  films  per  image  are  often  required;  (c)  the  FBDM  creates  images  with 
up  to  14  bits,  while  the  laser  printer  prints  8-bit  or  12-bit  images  according  to  its  look-up  tables; 
and  (d)  the  laser  film  printed  image  cannot  be  interactively  adjusted  for  such  image  enhancement 
as  window  and  level  settings.  No  study  has  been  conducted  in  developing  the  best  parameters  for 
the  use  of  laser  printed  films  for  FBDM. 

The  advantages  of  the  interactive  grayscale  workstation  are  the  following  (a)  the  ability  to 
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interactively  modify  the  display  image  throughout  the  12  bit  range  (window-level,  zoom  image 
processing,  computer-aided  diagnosis  algorithms);  (b)  the  use  of  multiple  displays  for  comparing 
images  (current  and  previous  examinations);  (c)  rapid  retrieval  and  display  from  the  archiving 
storage;  and  (d)  design  and  use  of  individual  display  protocols.  The  disadvantages  of  the 
interactive  grayscale  workstation  for  digital  mammography  are  the  following:  (a)  it  is  only 
possible  to  view  a  2k  x  2.5k  portion  of  the  full  resolution  FBDM  image;  (b)  sub-sampling  or 
binning  is  required  to  display  the  full-size  digital  mammography  image;  (c)  multiple  monitors  are 
required  for  each  interactive  workstation  to  display  the  two  CC  views  and  the  two  MLO  views; 
and  (d)  user  throughput  is  dependent  on  reader  facility  with  the  workstation  interactive  functions. 
No  studies  have  been  conducted  to  determine  the  best  parameters  for  an  interactive  FBDM 
grayscale  workstation. 

Models  of  individual  clinical  observations  permit  predicting  future  impact  of  new 
technologies  under  study.  A  stochastic  model  for  the  general  theory  of  screening  for  early 
detection  of  breast  cancer  was  first  presented  by  Eddy  (9).  His  model  utilized  the  metrics  of 
sensitivity  and  prognosis  as  a  function  of  early  detection.  His  work  provided  a  time-varying 
Markov  Chain  (10)  and  methods  for  estimating  the  probability  transitions  between  the  states  of 
the  model.  Others  (11-13)  have  added  to  the  information  concerning  stochastic  breast  cancer 
screening  methods,  modeling  the  disease  and  its  detection  via  one  of  more  diagnostic 
examinations.  There  exists  a  need  to  develop  a  screening  model  that  will  incorporate  the  FBDM 
technology  and  the  operational  parameters  of  display  technology,  together  with  the  metrics  of 
diagnostic  accuracy  and  throughput-to-cost-ratio.  Diagnostic  accuracy  determines  how  well  a 
screening  mammography  examination  is  used  for  detection  of  breast  cancer.  Throughput  is  the 
rate  at  which  mammography  examinations  can  be  completed.  The  cost  of  a  digital 
mammography  examination  is  calculated  from  the  resources  used  to  complete  the  examination. 
The  throughput-to-cost-ratio  is  a  metric  by  which  we  can  measure  the  cost  of  achieving  a  selected 
throughput.  No  stochastic  screening  model  has  been  implemented  which  utilizes  the  metrics  of 
diagnostic  accuracy  and  throughput-to-cost-ratio  in  optimizing  display  protocols  for  FBDM 
systems. 

Models,  due  to  their  assumptions,  have  restrictions  placed  upon  the  variables  being 
modeled.  Such  models  require  estimation  only  of  the  parameters  assumed  by  the  model.  This 
reduces  the  required  sample  sizes.  However,  concern  exists  regarding  the  model  errors  (all 
models  are  in  error).  The  use  of  a  model  requires  some  level  of  validation  or  experimental 
justification. 

MODELING 

A  good  model  of  an  examination  will  predict  the  usefulness  of  the  exam.  The  parameters 
of  the  model  will  have  to  be  estimated.  A  good  model  will  reduce  the  data  required  to  determine 
diagnostic  accuracy,  throughput,  and  cost. 

1.  ROC  Analysis 
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Analysis  is  based  upon  a  two-by-two  contingency  table  (Figure  1).  The  symbols  are:  ab  is 
abnormal  and  nl  is  normal;  AB  is  normal  and  NL  is  normal.  We  note  that  P(AB/ab)  +  P(NL/ab) 

=  1  and  that  PAB/nl)  +  P(NL/nl)  =  1 .  Thus,  of  the  four  conditional  probabilities,  two  of  them  are 
linearly  independent.  An  ROC  analysis  is  conducted  by  estimating  two  conditional  probabilities, 
based  upon  an  observation  variable  (Figure  2).  The  ROC  graph  (Figure  3)  is  generated  by 
moving  the  value  of  the  decision  threshold,  X^.  across  the  range  of  the  observational  variable  Xp 

The  steps  in  conducting  an  ROC  analysis  are  well  known  (14-21).  The  2x2  contingency 
table  is  carefully  defined.  A  statistical  power  is  initially  selected  and  the  number  of  samples  and 
readers  are  calculated.  The  number  of  samples  (cases)  are  selected  so  that  one-third  are  very 
difficult  to  read,  one-third  are  very  hard  to  read,  and  one-third  are  fairly  hard  to  read.  A  5-point 
multiscale  reading  is  performed  on  each  case  by  each  reader  for  specific  image  features.  There 
are  a  number  of  available  ROC  software  programs  that  will  process  the  multiscale  reading  data  to 
provide  the  ROC  graph.  The  area  under  each  reader’s  ROC  graph  is  the  index  of  performance 
(the  probability  that  a  random  selected  patient  undergoing  the  results  of  the  examination  will  be  a 
correct  true-positive). 

The  advantages  of  ROC  analysis  are  several.  It  is  a  well-known  tool,  often  used  by 
radiologists.  Software  exists  which  will  generate  the  ROC  graph  from  multi-scale  readings  by 
each  reader.  The  area  under  the  ROC  graph  is  used  as  an  index  of  reader  performance.  The 
disadvantages  of  ROC  analysis  is  that  it  remains  a  binary  decision  model.  It  is  difficult  to 
incorporate  population  statistics  (a  prior).  The  ROC  analysis  is  sensitive  to  modifying  the 
readers  or  cases. 

2.  Matrix  Modeling 

A  binary  examination  described  by  a  two-by-two  contingency  table  may  be  placed  into  a 
matrix  model  (Figure  4).  The  binary  decision  model  as  illustrated  has  two  input  states  (normal, 
abnormal)  and  two  output  states  (normal,  abnormal).  The  advantage  of  the  matrix  representation 
is  the  useful  manipulation  of  the  matrices  in  describing  probablistic  outcomes  of  multiple 
examinations.  For  example.  Figure  5  illustrates  the  results  of  cascading  two  examinations  while 
Figure  6  illustrates  a  decision  tree  model  of  the  two  cascaded  examinations.  The  results  of 
cascading  two  examinations  show  that  the  resultant  transition  probabilities  are  less  than  that  of 
each  individual  examination  -  a  result  sometimes  called  “leading  information,”  In  Figure  6,  the  a 
priori  probabilities  are  P(l)  and  P(2).  For  example  P(7/l)  is  given  by:  P(7/l)  =  P13P57  +  P14P67  = 


A  model  of  double  reading  is  illustrated  in  Figure  7.  In  double  reading,  reader  1  and 
reader  2  independently  read  the  results  of  the  examination.  They  then  compare  their  outcomes 
using  a  decision  rule.  Table  1  defines  the  set  of  possible  reader  outcomes  for  each  reader.  Tables 
2  and  3  provide  a  decision  rule  that  defines  the  conditions  under  which  the  readers  are  said  to  be 
equal  or  different.  A  more  conservative  decision  rule  would  be  to  say  that  the  two  readers  are 
equal  if  they  achieve  identical  outcomes  and  that  the  two  readers  are  different  if  they  do  not  have 
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identical  outcomes.  It  is  believed  by  some  researchers  that  computer  aided  decision  algorithms 
could  serve  as  a  partial  second  reader. 

The  sequence  of  outcomes  from  the  decision  rule,  based  upon  the  results  of  both  readers, 
can  be  assumed  to  be  an  n-sequence  of  the  symbol  set  (0,1)  where  the  alphabet  symbol  of  “0” 
means  equal  results  and  the  alphabet  symbol  of  “1"  means  different  results  from  the  two  readers. 
If  these  results  are  observed  for  a  very  long  time,  the  sequence  of  symbols  (with  probability  one) 
will  be  “typical.”  That  is,  the  resultant  sequence  of  outcomes  from  the  decision  rule  is  ergodic. 
This  implies  that  a  n-sequence  of  outcomes,  as  n  increases,  will  result  in  typical  sequences  (those 
close  to  the  expected  value)  and  the  rest  being  non-typical  (those  with  probabilities  approaching 
zero  as  n  increases) 

3.  Uncertainty  Modeling 

Uncertainty  modeling  is  a  probablistic  scheme  (Figure  8)  in  which  there  is  a  sequence  of 
trials  (say  n)  and  conclusions  are  drawn  on  the  resultant  n-sequences  being  close  to  there 
expected  values  (22).  The  concept  of  typical  sequence,  as  introduced  by  Wolfowitz  (23),  who 
called  them  X  sequences,  and  as  used  by  Ash  (24),  is  related  to  the  asymptotic  behavior  of  its 
independent,  identically  distributed  random  variables  (25-28).  Let  Y  be  the  random  variable 
taking  on  the  values  of  (  y2)(for  the  outcome  of  the  FBDM  test)  or  (y„  y2  yj  y^,  y^)  (for  the 
outcome  of  the  readers  from  a  multiscale  reading,  respectively).  Suppose  that  the  experiment 
associated  with  Y  is  performed  “n”  times.  That  is  ,  a  sequence  will  be  generated  of  Y,,  Yj, ...  Y„ 
of  independent,  identically  distributed  random  variables,  each  having  the  same  probability 
distribution  as  Y.  If  we  define  a  fimction  fi=fi(Y„  Yj, ...,  YJ  to  be  the  number  of  times  the 
symbol  y^  occurs  in  the  sequence  Y„  Y2, ...  Y„,  then  yj  will  f;  has  a  binomial  distribution  with 
parameters  n  and  pj.  We  identify  a  “typical”  sequence  as  those  sequences  in  which  fj  is  “close”  to 
npi  for  every  i  where  np;  is  the  number  of  times  that  the  expected  value  of  yj  will  occur  in  the  n- 
sequence.  Then  we  note  that  the  set  of  nontypical  sequences  has  a  small  probability  of 
occurrence.  Thus  in  a  typical  sequence,  each  symbol  y^  occurs  approximately  with  its  expected 
frequency  of  npj.  The  formal  definition  is  the  following.  Given  c>0,  choose  any  positive  number 
k  such  that  1/k)^  <^/M  where  M  is  the  number  of  symbols  or  values  that  the  random  variable  Y 
takes  on.  Let  b  =  (bj,  b2, ...,  b^)  be  a  sequence  of  symbols,  each  bi  being  one  of  the  symbols  of 
the  set  (Y„  Y2, ...  Y^).  We  say  the  sequence  b  is  typical  if 

/fi(b)  -  npi  /x/  (npi  (1-pi)]  kk 

It  can  be  shown  that  the  following  is  true:  (a)  the  set  of  nontypical  sequences  of  length  “n”  has  a 
total  probability  <c;  and  (b)  the  number  of  typical  sequences  of  length  “n”  is  2”^^^"n\  where  r„ 
approaches  zero  exponentially  as  n  grows  large  and  (c), 

H  =  (fi(b)log  p,  +  f2(b)  log  P2  +  ...+  fM(b)  log  p  J 

If  the  output  symbols  are  Xj,  X2, ...,  X^,  and  the  out  symbols  are  y„  y2, ...,  yL  and  the 
transition  probabilities  are  [a,j],  where  ajj  =  P(yj/Xi),  i  =  1, ...,  M  and  j  =  1, ...,  L,  then  the  joint 
distribution  of  the  input  X  and  output  Y  is  given  by 

P(X  =  Xj,  Y  =  yj)  =  P(Xi)P(yj/Xi),  i  =  1, 2, ...,  M,  and  j  =  1, 2, ...,  L 
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and  the  distribution  of  Y  is  given  by 

P(yj)  =  P(X,)P(yj/x,)  +  P  (x2)P(y/x2)  +  ...  +P(xM)P(y/x^). 

For  M=2  and  L=5,  then  P(Y  =  yj)  =  P(x,)P(yj/x,)  +  P(x2)P(yj/x2),  j  =  1, ...,  5.  If  the  experiment  is 
repeated  “n”  times,  then  if  b  =  (b„  b2, ...,  bj  is  a  sequence  of  resultant  n-symbols,  each  bj  being 
one  of  the  elements  (y,,  y2, ...,  ys),  then  we  say  that  the  n-sequence  is  typical  if  it  satisfies  the 
above  definition. 

The  significance  of  the  use  of  typical  sequences  is  in  analyzing  the  model  of  the 
diagnostic  test  and  reader  responses.  Typical  sequences  reduce  the  number  of  n-sequences  that 
require  analysis.  For  a  large  n  (number  of  independent  trials),  there  are  approximately 
typical  input  n-sequences,  each  with  probability  roughly  Similarly  ther  are  2"“^^^^  typical 

output  n-sequences  and  2"”^’^^  typical  pairs  of  input  and  output  n-sequences.  A  typical  pair  may 
be  generated  by  first  selecting  a  typical  output  n-sequence  y  and  then  selecting  a  typical  input  n- 
sequence  x  such  that  (x,y)  is  a  typical  pair.  Since  the  number  of  typical  output  sequences  is  2"”^^ 
and  the  number  of  typical  pairs  is  then  for  each  typical  output  sequence  y,  there  are 

2nH(X,Y)-H(Y)  _  2nH(X/Y) 

input  sequences  x  such  at  (x,y)  is  a  typical  pair.  That  is,  if  a  typical  sequence  y  is  presented  then 
the  number  of  typical  input  sequences  possible  are  approximately  2"^^^  (not  2"”^^^)  each  with 
approximately  the  same  probablity,  2'"”^^.  This  is  important  since  it  reduces  the  number  of  n- 
sequences  that  are  to  be  analyzed.  Hence,  if  the  experiment  is  conducted  say  100  times  (n=100 
or  reading  out  100  cases)  then  we  can  expect  approximately  2'°°’^^'^^  typical  sequences,  each  with 
approximate  probability  of  It  is  known  that,  given  the  transition  probabilities  (such  as 

obtained  from  an  ROC  analysis),  then  by  varying  the  input  distribution  P(X),  one  can  obtain  a 
maximum  of  the  mutual  information,  I(X;Y)  =  H(X)  +  (H(Y)  -  H(X/Y),  between  the  input,  X, 
and  the  output,  Y,  which  is  called  the  channel  capaeity,  C.  Hence,  given  the  transition 
probabilities  between  input  and  output,  (Pjj),  there  exists  an  input  distribution  P(X)  such  that  the 
mutual  information  between  X  and  Y,  I(X;Y),  is  a  maximum.  The  result  suggests  that  for  a 
given  input  distribution  P(X),  the  a  priori  distribution,  the  transition  probabilities  may  be 
changed  to  better  approach  the  channel  capacity,  C. 

If  we  are  modeling  an  examination,  then  Figure  9  illustrates  the  relations  between 
different  uncertainty  measures.  These  measures  are  given  by: 

H(X)  =  -£Pi  log  Pi 

H(Y)  =  -i;PjlogP, 
and 

H(X,Y)  = -E,  P(X„  Yj)  log  P  (Xi,  Yi) 

The  mutual  information  between  (X,Y)  is  given  by 

I(X;Y)=H(X)-HpC/Y) 

=  L  Ij  P(Xi,  Yj)  log  P(Xi,  Yi) 

P(Xi) 

=i:i  Zj  P(Xi,  Yj)  log  P(Xi,Yi) 

P(Xi)  P  (Yj) 
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Figure  9  illustrates  a  graphical  schematic  of  these  relationships. 

Figure  10  is  a  model  of  double  reading  as  previously  cited.  We  are  interested  in  the 
mutual  information,  I  (A;  B,  C,  D,  E).  This  is  the  mutual  information  regarding  “A”  on  condition 
of  the  occurrence  of  (B,  C,  D,  E).  It  is  given  by: 

I  (A;  B,  C,  D,  E)  =  I  (A/C)  + 1  (A;  B/C) 

+ 1  (A;  D/B)  I  (A;  D/C) 

+ 1  (A;  E/B)  + 1  (A;  E/C) 

+ 1  (A;  E/D). 

The  conditional  mutual  information  terms,  such  as  I  (A;  D/C),  is  the  average  amount  of 
information  about  the  set  “A”  provided  by  an  observation  from  the  set  “D”  after  an  observation 
from  the  set  “C”. 

Figure  1 1  is  a  model  for  display  protocols  (either  laser  printed  film  or  soft  copy,  the 
grayscale  workstation)  for  digital  mammography.  Then  the  occurrence  of  (B,  C,  D,  E,  F,  G)  that 
tells  us  about  the  set  “A”  is  the  following: 

I  (A;  B,  C,  D,  E,  F,  G)  =  I  (A;  C)  + 1  (A;  B/C) 

+ 1  (A;  D/B)  + 1  (A;  D/C)  +  J  (A;  E/B) 

+ 1  (A;  E/C)  + 1  (A;  E/D)  + 1  (A;  F/B) 

+ 1  (A;  F/C)  + 1  (A;  F/D)  + 1  (A;  F/E) 

+ 1  (A;  G/B)  + 1  (A;  G/C)  + 1  (A;  G/D) 

+ 1  (A;  G/E)  + 1  (A;  G/F). 

The  necessary  parameters  for  this  model  are  conditional  probabilities.  The  value  of  I  (A; 
B,C,D,E,F,G)  tells  us  the  number  of  typical  sequences  in  an  n-sequence  as  well  as  their 
probability  of  occurrence.  The  decomposition  of  I  (A;  B,  C,  D,  E,  F,  G)  into  simpler  terms,  such 
as  I  (A;  G/F)  provides  for  a  much  simpler  model. 

THROUGHPUT  ANALYSIS 

The  three  techniques  for  performance  measurement  of  a  system  are  analytical  modeling, 
simulation,  and  measurement  of  an  actual  system  (29).  The  major  consideration  in  deciding 
which  evaluation  technique  to  use  is  the  life-cycle  stage  in  which  the  system  resides. 
Measurements  are  possible  if  technology  similar  to  the  system  being  evaluated  already  exists.  If 
the  system  under  study  is  new,  then  only  analytical  modeling  and  simulation  methods  are 
possible.  Simulation  is  difficult  and  time-consuming  to  accomplish.  For  the  mammography 
systems,  the  only  reasonable  evaluation  technique  is  analytical  modeling.  The  performance 
metrics  being  used  in  our  modeling  of  these  systems  are  throughput  and  diagnostic  accuracy. 
Throughput  is  defined  as  the  rate  (jobs-per-minute)  at  which  requests  can  be  serviced  by  the 
system.  The  throughput  of  a  system  generally  increases  as  the  load  on  the  system  initially 
increases,  without  the  addition  of  more  resources;  after  a  certain  load,  the  throughput  stops 
increasing  due  to  one  or  more  bottlenecks.  Throughput  analysis  is  accomplished  by  listing  the 
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resources  used  and  the  steps  required  to  complete  one  job.  The  resource  with  the  highest  total 
service  demand  has  the  highest  utilization  and  is  called  the  bottleneck  resource.  The  area  under 
the  ROC  curve  (14)  is  the  probability  of  correctly  distinguishing  -  on  the  basis  of  the  diagnostic 
test  results  alone  -  a  randomly  selected  disease  case  from  a  randomly  selected  non-diseased  case. 

Table  4  and  table  5  (30)  are  the  resource  utilization  tables  generated  for  two  throughput 
and  bottleneck  analyses.  The  top  row  of  each  table  lists  the  resources  being  modeled.  The  rows 
of  tables  are  the  steps  required  to  complete  one  reading.  The  bottom  rows  (throughput  per 
minute)  are  generated  by  alternately  assuming  that  each  resource  in  the  columns  is  operating  as 
rapidly  as  it  possibly  can. 

The  bottleneck  einalysis  described  above  is  based  on  Little's  Law  (3 1)  of  mean  value 
queuing  analysis.  Little's  Law  states  that  the  mean  number  of  jobs  in  a  system  equals  the  mean 
throughput  rate  multiplied  by  the  mean  time  in  the  system.  We  have  conducted  similar  studies 
for  screen-film  mammography.  Table  r  (labeled  "MCV")  is  a  Resource  Utilization  table  for 
mammography  at  the  Medical  College  of  Virginia  (provided  by  Dr.  Ellen  Shaw  de  Paredes)  and 
Table  5  (labeled  "SP")  is  for  an  outpatient  Breast  Imaging  Center. 

The  throughput  analysis  is  condueted  by  obtaining  the  upper  and  lower  bound  (Figure  12) 
on  the  actual  throughput  as  a  function  of  one  or  more  of  the  resources.  Upper  and  lower  bounds 
are  required  to  understand  the  otherwise  complex  analysis  of  the  actual  throughput.  If  the 
radiologist  is  selected  as  the  independent  variable  (increasing  the  number  of  radiologists),  then 
the  upper  bound  on  the  throughput  is  given  by 

Upper  Bound  =  min[l T,y„  N/(T,y,  + 

where  T^y^  is  the  time  used  by  the  system  less  the  radiologist  and  resident  time  and  Trader  is  the 
time  used  by  the  radiologist  and  the  resident  reading  out  the  cases  (while  system  is  waiting). 

Then  l/T^ys  is  the  maximum  throughput.  The  term  N(Tsys  +  Trader)  relating  increases  in  the 
number  of  radiologists  (and  residents)  increases  linearly  with  N  until  it  reaches  l/T^ys,  after  which 
the  system  cannot  increase  its  throughput  no  matter  how  many  radiologists  (and  residents)  are 
added  to  the  reading  room.  This  is  the  case  when  the  radiologist  is  unable  to  do  any  other  work 
above  the  throughput  limit  because  the  system  is  then  queuing  examinations.  We  note  that  as  N 
inereases  (N  could  be  other  resources  such  as  technologists),  then  there  is  a  method  for 
calculating  the  lower  bound  on  the  throughput.  That  is. 

Lower  Bound  of  Throughput  =  N/(NTsys  +  T^eader)- 
This  will  occur  when  the  system  is  busy  doing  other  things  and  the  newest  case  is  last  in  the 
queue  to  receive  service.  Figure  22  illustrates  this  analysis  of  throughput. 

SUMMARY 

We  have  reviewed  modeling  of  mammography  examinations.  Modeling  is  essential  due 
to  the  large  number  of  variables.  However,  a  carefully  selected  model  will  require  only  the 
estimation  of  the  model  parameters.  Of  the  models  we  have  described,  the  uncertainty  modeling 
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is  superior  owing  to  its  restrained  relationships  and  requiring  only  the  estimation  of  conditional 
probabilities.  The  throughput  analysis  is  a  method  that  is  simple  to  utilize  in  estimating 
workflow. 

CONCLUSIONS 

The  following  conclusions  regarding  the  Telemammography  research  project  have  been 
reached: 

1 .  Collection  of  the  400  mammography  screen-film  database  is  critical  to  the  project. 

2.  The  grayscale  workstation  must  be  carefully  implemented  to  perform  a  ROC  softcopy 
analysis.  Critical  elements  are  the  following: 

a.  Needed  software  to  obtain  an  acceptable  display  protocol. 

b.  Required  disc  and  ROM  space  to  archive  four  4k  x  4k  x  12  bit  images. 

c.  Need  CD-ROM  to  archive  50  cases  for  each  reader  session. 

d.  Need  display  software  that  will  provide  the  ability  to  read  40  cases  per  hour. 

3.  Conclusions  regarding  an  adequate  wide-area-network  await  completion  of  the  fourth 
year  of  the  project.  The  following  are  being  tested. 

1 .  T1  Link. 

b.  ISDN  Link. 

c.  FRAME  RELAY  Link. 

d.  Satellite  Link. 

LAY  PUBLIC  ABSTRACT 

Telemammography  is  the  transmission  and  interpretation  of  digitized  mammographic 
images.  The  use  of  telemammography  will  allow  for  interpretation  of  on-site  mammograms 
from  distant  areas  by  expert  radiologists  at  a  central  site.  This  can  potentially  increase  the  access 
to  mammography  for  many  women. 

We  are  testing  a  telemammography  system  utilizing  digitized  mammograms  that  are 
transmitted  through  telephone  lines  or  satellites  and  are  interpreted  on  computer  workstations. 
The  goal  of  this  study  is  to  determine  the  requirements  to  deliver  high  quality  mammographic 
images  from  remote  locations. 

To  test  the  system  we  have  collected  400  normal  and  abnormal  mammograms  which  have 
been  interpreted  by  12  radiologists.  The  radiologists  are  interpreting  the  studies  on  film  as  well 
as  computer  workstations,  and  we  are  comparing  their  accuracy  with  two  methods. 
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We  also  are  assessing  the  performance  of  the  system  by  comparing  the  patient  workflow 
when  their  mammograms  are  performed  using  routine  mammographic  film  versus  digitized 
mammograms. 

We  have  found  that  reading  mammograms  on  the  workstations  versus  on  film  reduces  the 
patient  throughput.  Also  because  of  the  decreased  brightness  of  images  on  the  workstations 
versus  viewing  films  on  traditional  light  boxes,  the  display  of  some  images  is  difficult. 
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Figure  1 

Two-By-Two  Contingency  Table 
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Figure  2 

Model  of  ROC.  The  value  of  Xc  is  moved  across  the  range  of  the 
observation  variable,  generating  the  ROC  graph 
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Figure  3 
ROC  Graph 
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Input  Output 


P  j  1  =  P(AB/ab)  Pi2  =  P(NL/ab) 

P21  =  P(AB/nl)  P22  =  P(NL/nl) 


Figure  4 

Matrix  Modeling 
of  an  Examination 
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Model  of  Double  Reading 


Table  I 

Reader  Outcome  for 
Modeling  of  Double  Reading 


Outcome  States 

7  or  15 

8  or  16 

9  or  17 

10  or  18 

11  or  19 

12  or  20 


Meaning  Assigned 
Patient  recalled 

Patient  negative  (return  1  year) 

Benign  findings  (return  1  year) 

Probably  benign  (return  6  month) 
Suspicious  finding  (do  biopsy) 
Malignant,  highly  suspicious  (do  biopsy) 
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Table  2 

Decision  Rule  for  Two  Readers  Outcomes  Said  to  Be  Equal 


Outcome: 

•  Negative  and  benign 

•  Suspicious  and  malignant 

•  Probably  benign 
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Table  3 

Decision  Rule  for  Two  Readers  Outcomes  Said  to  Be  Different 


Outcome: 

•  Benign  and  probably  benign 

•  Probably  benign  and  suspicious  finding 

•  Negative  and  suspicious  finding 

•  Benign  and  suspicious 

•  Negative  and  malignant 

•  Benign  and  malignant 

•  Probably  benign  and  malignant 
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X.  Input 


Y.  Output 
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H(X)  H(X,Y)  H(Y) 

I(X;Y)  =  H(X)-H(X/Y) 

=  H(X)  =  H(Y)-H(X,Y) 


Figure  8 

Uncertainty  Measures 
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H(Y) 


I(X;Y) 


Figure  9 

Uncertainty  Measures 


30 


Exam 


I 


Figure  10 

Modeling  of  Double  Reading 


31 


Number  of  Jobs 

Y  =  (’T  -f-  T  ^  T 

reader  ^system/  ^system 


Figure  12 

Throughput  Analysis 
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Reader02  Digital  ROC:  Type=Diag  Area=.6949  STD  Area=.0398 
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